• Add Playwright harnesses that boot backend + frontend dev servers in-process, off-cluster.
• Provide legacy Tier B harness to run existing specs unchanged with Scalprum dynamic plugins.
• Document setup, limitations, and add guest-auth + in-memory SQLite config overlay.

graph TD
  PW(["Playwright runner"]) --> CH{"Harness"} -->|"Legacy Tier B"| LEG(["Legacy app dev server (janus-cli)"]) --> BR["Browser-driven E2E"]
  CH -->|"app-next"| NEXT(["App-next dev server (backstage-cli)"]) --> BR
  PW --> BE(["Backend dev server (backstage-cli)"]) --> BR
  CFG[["app-config.local-e2e.yaml"]] --> BE
  CFG --> LEG
  CFG --> NEXT
  DPR[("dynamic-plugins-root")] --> LEG
  subgraph Legend
    direction LR
    _svc(["Dev server"]) ~~~ _file[["Config file"]] ~~~ _db[("Plugins root")] ~~~ _dec{"Decision"} ~~~ _box["Test target"]
  end

Loading

The following are alternative approaches to this PR:

1. Single Playwright config with projects (legacy + app-next)

• ➕ Eliminates duplication of reporters/timeouts/webServer wiring
• ➕ Allows running one or both harnesses via Playwright project selection
• ➕ Centralizes env handling (PATH, CI reuseExistingServer)
• ➖ Slightly more complex conditional webServer setup
• ➖ Harder to keep legacy testMatch constraints isolated without care

2. Use container-based rhdh-local for local E2E

• ➕ Closest to production image runtime; fewer dev-server quirks
• ➕ Avoids needing local workspace CLIs on PATH
• ➖ Requires container runtime and image pulls
• ➖ Heavier/slower for automated E2E; contradicts ‘cluster-free/in-process’ goal

Recommendation: Keep the PR’s approach (in-process dev servers via Playwright webServer) because it directly optimizes for fast, cluster-free automated E2E and preserves existing legacy specs unchanged. If this becomes long-lived, consider consolidating common Playwright config into a shared base and using Playwright projects to reduce config duplication while keeping distinct legacy vs app-next server commands.

Tests (3) +248 / -0

playwright.app-next-local.config.tsAdd Playwright config to boot backend + app-next dev server +94/-0

Add Playwright config to boot backend + app-next dev server

• Defines a Playwright configuration that starts the backend and packages/app-next dev server via webServer, reusing servers locally and forcing fresh starts in CI. Ensures repo-root .bin CLIs are discoverable via PATH and sets reporting/timeout defaults for the harness.

e2e-tests/playwright.app-next-local.config.ts

playwright.legacy-local.config.tsAdd Tier B Playwright config for legacy app with dynamic plugins +105/-0

Add Tier B Playwright config for legacy app with dynamic plugins

• Creates a legacy harness config that runs a curated subset of existing specs off-cluster. Boots backend and packages/app via backstage-cli/janus-cli with layered configs including dynamic plugins and the local E2E overlay, and reuses servers outside CI.

e2e-tests/playwright.legacy-local.config.ts

guest-identity.spec.tsAdd app-next local guest sign-in + identity smoke test +49/-0

Add app-next local guest sign-in + identity smoke test

• Introduces a minimal E2E spec validating the app-next harness: guest sign-in succeeds, core navigation is present, and the Settings page renders the guest Backstage identity. Assertions intentionally match app-next UI behavior rather than legacy home-page expectations.

e2e-tests/playwright/app-next-local/guest-identity.spec.ts

Documentation (1) +105 / -0

local-e2e-harness.mdDocument cluster-free local E2E harness usage and constraints +105/-0

Document cluster-free local E2E harness usage and constraints

• Adds a dedicated guide describing the legacy Tier B and app-next harnesses, dynamic-plugins-root population paths, and what’s currently verified. Documents key limitations (global-header mounting, app-next dynamic plugin loading, and external-service-dependent specs) and explains why this approach differs from rhdh-local.

docs/e2e-tests/local-e2e-harness.md

Other (2) +28 / -1

app-config.local-e2e.yamlAdd local E2E config overlay (guest auth + in-memory DB) +25/-0

Add local E2E config overlay (guest auth + in-memory DB)

• Introduces a shared config overlay for cluster-free E2E runs. Enables the guest auth provider (including CI-safe allowance) and forces an in-memory better-sqlite3 backend database for fully self-contained runs.

app-config.local-e2e.yaml

package.jsonAdd yarn scripts to run legacy-local and app-next-local harnesses +3/-1

Add yarn scripts to run legacy-local and app-next-local harnesses

• Adds two Playwright entrypoint scripts for the new harness configs. Keeps existing tooling scripts intact while enabling one-command local harness execution.

e2e-tests/package.json

Description

With reuseExistingServer enabled outside CI, Playwright will reuse any process responding on the
configured localhost URLs, even if it's not the intended RHDH harness. Because the URLs are
hard-coded to localhost:3000 and localhost:7007, this can lead to tests running against
stale/foreign servers and producing misleading results.

Code

e2e-tests/playwright.legacy-local.config.ts[R88-100]

+      url: backendReadiness,
+      reuseExistingServer: !process.env.CI,
+      timeout: 180 * 1000,
+      stdout: "pipe",
+      stderr: "pipe",
+    },
+    {
+      command: `janus-cli package start ${sharedConfigArgs}`,
+      cwd: "../packages/app",
+      env: { ...process.env, PATH: `${repoRootBin}:${process.env.PATH}` },
+      url: frontendUrl,
+      reuseExistingServer: !process.env.CI,
+      timeout: 240 * 1000,

Relevance

⭐⭐ Medium

No prior review evidence on reuseExistingServer risks; pattern likely tolerated for local dev but
could be tightened.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

Both configs hard-code the target URLs/ports and enable reuse of any existing server when CI is
not set, which makes it possible to run tests against an unintended process bound to those ports.

e2e-tests/playwright.legacy-local.config.ts[31-33]
e2e-tests/playwright.legacy-local.config.ts[88-100]
e2e-tests/playwright.app-next-local.config.ts[24-26]
e2e-tests/playwright.app-next-local.config.ts[75-89]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The harness sets `reuseExistingServer: !process.env.CI` while targeting fixed `http://localhost:3000` and `http://localhost:7007/...`. If anything else is listening on those ports (or a previous run is still alive), Playwright may attach to the wrong process.

### Issue Context
This affects both `playwright.legacy-local.config.ts` and `playwright.app-next-local.config.ts`.

### Fix
Pick one:
1) Make reuse explicit (opt-in), e.g. `reuseExistingServer: process.env.REUSE_EXISTING_SERVER === 'true'`.
2) Use per-run ports (random/free port) and pass them into backend/frontend via config/env overrides, and derive `baseURL`/readiness URLs from those ports.
3) Add a lightweight identity check before reusing (e.g., fetch a known endpoint and validate a header/body) and fail if it doesn’t match.

### Fix Focus Areas
- e2e-tests/playwright.legacy-local.config.ts[31-33]
- e2e-tests/playwright.legacy-local.config.ts[88-100]
- e2e-tests/playwright.app-next-local.config.ts[24-26]
- e2e-tests/playwright.app-next-local.config.ts[75-89]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

The local harness webServer env sets NODE_OPTIONS to a fixed value, overwriting any pre-existing
NODE_OPTIONS from the environment. This can break harness startup in CI/dev setups that rely on
other Node flags (e.g., memory limits or preload hooks).

Code

e2e-tests/playwright.legacy-local.config.ts[R83-87]

+      env: {
+        ...process.env,
+        PATH: `${repoRootBin}:${process.env.PATH}`,
+        NODE_OPTIONS: "--no-node-snapshot",
+      },

Relevance

⭐ Low

Repo already sets NODE_OPTIONS="--no-node-snapshot" unconditionally in Docker/CI; clobbering appears
accepted.

PR-#2297

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

Both harness configs spread ...process.env but then unconditionally override NODE_OPTIONS, which
discards any upstream flags.

e2e-tests/playwright.legacy-local.config.ts[83-87]
e2e-tests/playwright.app-next-local.config.ts[70-74]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The Playwright `webServer.env` blocks replace `NODE_OPTIONS` with `"--no-node-snapshot"`, which drops any existing `NODE_OPTIONS` settings inherited from the parent environment.

### Issue Context
This affects both the legacy and app-next local harness configs.

### Fix
Construct `NODE_OPTIONS` by appending `--no-node-snapshot` to any existing value, e.g.:

```ts
const nodeOptions = [process.env.NODE_OPTIONS, '--no-node-snapshot']
 .filter(Boolean)
 .join(' ');

NODE_OPTIONS: nodeOptions,
```

### Fix Focus Areas
- e2e-tests/playwright.legacy-local.config.ts[83-87]
- e2e-tests/playwright.app-next-local.config.ts[70-74]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

@@            Coverage Diff             @@
##             main    #5005      +/-   ##
==========================================
- Coverage   55.39%   54.77%   -0.62%     
==========================================
  Files         122      110      -12     
  Lines        2365     2147     -218     
  Branches      563      538      -25     
==========================================
- Hits         1310     1176     -134     
+ Misses       1049      969      -80     
+ Partials        6        2       -4     

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5c9e678...ee3120b. Read the comment docs.

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.